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Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent tenn adjustment. See 37 CFR 1.704(b). 

Status 

1 )K Responsive to connmunication(s) filed on 23 April 2004 . 
2a)K This action is FINAL. 2b)n This action is non-final. 

3) 0 Since this application is in condition for allowance except for formal nnatters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim{s) 52-89 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) ^ Claim(s) 52-89 is/are rejected, 

7) n Claim(s) is/are objected to. 

8) n' Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) n The specification is objected to by the Examiner. 

10)0 The drawing(s) filed on is/are: a)^ accepted or b)\Z\ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 .121 (d). 
11 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 
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1 0 Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 
Response to Arguments 

1 . Applicant's arguments filed 23 April 2004 have been fully considered but they are 
not persuasive. 

The applicant argues on page 16 second paragraph, that Kaiser does not teach 
or suggest receiving a "viewer actuation of the mask overlay as recited in the amended 
independent claims 52, 55, 588, 61, 64, 67, 70, and 73, in that the amended claims 
require more than just an identification of a mask overlay. The examiner disagrees. 
Kaiser teaches a user selection of an action, which directly corresponds to a mask 
overlay. Accordingly, the user indirectly (by means of a viewer actuation of an action) 
actuates one of the mask overlays of the video streams via a selection device (1 700) 
(col. 5, II. 39-42, col. 10, II. col. 10-11, II. 51-8). 

The applicant further argues on page 16 third paragraph through page 17 first 
paragraph, that there is no prior art indication to suggest increasing user interactivity. 
The examiner disagrees. Kaiser teaches providing interactive products to the viewer 
(col. 2, II. 23-32); the mere presence of an interactive product promotes interactivity for 
the user. 

The application argues on page 17 paragraph 2, that there is no indication in 
Kaiser to switch from one video stream to another in order to increase user interactivity. 
The examiner disagrees. Kaiser teaches providing additional information such product 
purchasing (col. 12, II. 29-54), which promotes user interactivity, one of ordinary skill in 
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the art readily recognizes that information can arrive in a plurality of forms, one of which 
is via a video stream as taught by Freeman. 

The applicant argues on page 17, third paragraph, that the system fails to 
provide "desirable programming to the user" to motivate a person of ordinary skill in the 
art to modify Kaiser. The examiner disagrees; Freeman teaches providing personalized 
information to the user (see Abstract). Accordingly, Freeman teaches providing 
desirable programming to the user. 



Drawings 

2. In addition to Replacement Sheets containing the corrected drawing figure(s), 
applicant is required to submit a marked-up copy of each Replacement Sheet including 
annotations indicating the changes made to the previous version. The marked-up copy 
must be clearly labeled as "Annotated Marked-up Drawings" and must be presented in 
the amendment or remarks section that explains the change{s) to the drawings. See 37 
CFR 1.121(d). Failure to timely submit the proposed drawing and marked-up copy will 
result in the abandonment of the application. 

3. In the amendment filed 23 April 2004 that applicant attached sheets of drawings 
including changes Fig. 2D (see page 14), however the attachments "Replacement 
Sheet" and "Annotated Sheet Showing Changes" are missing from the case, thereby 
need to be re-submitted. 



Claim Rejections - 35 USC § 103 
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4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 52-75 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
U.S. Patent 6,615,408 to Kaiser et al. in view of U.S. Patent 5,861,881 to Freeman et al. 

Regarding claims 52, 55, 58, 64, 67, 70, Kaiser teaches a visual highlight as 
shown in figure 6B, which reads on a mask generator for providing a plurality of 
overlays. Kaiser teaches a receiver for receiving the streams and receiving user input 
(fig. 1, label 1300). Further, Kaiser teaches placing plural placement zones regions in a 
plurality of frames thereby permitting a plurality of zones to be presented simultaneously 
(col. 6, II. 25-34). Further, Kaiser teaches receiving selection input identifying an 
overlay by selecting an action to be performed (col. 9, II. 48-65), but Kaiser is silent on 
switching from one video stream to a second stream. Freeman teaches in response to 
past user inputs, at trigger points inserting another video stream being concurrently 
broadcasted (col, 13. II. 28-45). Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Kaiser by switching 
video streams as taught by Freeman in order to increase user interactivity and provide 
desirable programming to the user. 

Regarding claims 53, 56, 59, 65, 68, 71, Kaiser teaches sending input to a 
remote location via a backchannel communications connection, as shown in figure 1 
with the Internet as the data network (col. 5, II. 45-54). 
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Regarding claims 54, 57, 60, 66, 69, 72, Kaiser teaches location and shape 
information of the regions (fig. 6B, col. 10, II. 20-41). 

Regarding claims 61 and 73, Kaiser is silent on a plurality of video stream 
identifiers associated with one of the plurality of video streams. Official Notice is taken 
that PIDs are well known in the art. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Kaiser by using 
PIDs in order to identify one of the plurality of program streams thereby enabling the 
receiving device to decode desirable packets of data. Kaiser teaches a visual highlight 
as shown in figure 6B, which reads on a mask overlay. Kaiser teaches placing plural 
placement zones regions in a plurality of frames thereby permitting a plurality of zones 
to be presented simultaneously (col. 6, II. 25-34). Kaiser teaches receiving selection 
input identifying an overlay by selecting an action to be performed (col. 9, II. 48-65). 
The mask has corresponding actions that can be taken (fig. 6C, 6D, label 6400), which 
equates to a first data structure element. Kaiser teaches an action resource retrieval 
step (col. 9, II. 48-65), which reads on a second data structure element, but is silent on 
switching from one video stream to a second stream. Freeman teaches in response to 
past user inputs, at trigger points inserting another video stream being concurrently 
broadcasted (col. 13, 11. 28-45). Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Kaiser by switching 
video streams as taught by Freeman in order to increase user interactivity and provide 
desirable programming to the user. 
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Regarding claims 62 and 74, Kaiser teaches location and shape information of 
the regions (fig. 6B, col. 10, II, 20-41). 

Regarding claims 63 and 75, Kaiser is silent on an audio and private data stream 
associated with the first one of the said plurality of video streams. Official Notice is 
taken the associating an audio and private data stream with video is well known in the 
art. Therefore, it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to modify Kaiser by associating an audio and private data 
stream with video in order to organize related data thereby enabling the receiver to 
efficiently access different streams. 

6. Claims 76-80 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
U.S. Patent 6,615,408 to Kaiser et al. in view of U.S. Patent 5,861 ,881 to Freeman et al. 
and U.S. Patent 5,818,438 to Howe et al. 

Regarding claim 76, Kaiser teaches a visual highlight as shown in figure 6B, 
which reads on a mask generator for providing a plurality of overlays. Kaiser teaches a 
receiver for receiving the streams and receiving user input (fig. 1 , label 1300). Further, 
Kaiser teaches placing plural placement zones regions in a plurality of frames thereby 
permitting a plurality of zones to be presented simultaneously (col. 6, II. 25-34), however 
Kaiser is silent on providing an indication of additional information in order to generate 
zones in response to a user command. Howe teaches providing an icon as an 
indication that additional content is available and in response to seeing the button, 
receiving a request to retrieve information (col. 4, II. 30-50, col. 29, II. 12-27). Therefore, 
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it would have been obvious to one of ordinary skill in the art at the time the invention 
was made to modify Kaiser by providing an indication of content and receiving user 
input to perform an action associated with the icon as taught by Howe in order to 
investigate subject matter more thoroughly and according to his or her own tastes 
(Howe: col. 2, II. 48-50). Further, Kaiser teaches receiving selection input Identifying an 
overlay by selecting an action to be performed (col. 9, II. 48-65), but Kaiser is silent on 
switching from one video stream to a second stream. Freeman teaches in response to 
past user inputs, at trigger points inserting another video stream being concurrently 
broadcasted (col. 13, II. 28-45). Therefore, it would have been obvious to one of 
ordinary skill In the art at the time the invention was made to modify Kaiser by switching 
video streams as taught by Freeman in order to increase user interactivity and provide 
desirable programming to the user. 

Regarding claim 77, Kaiser teaches an overlay associated with a first timing data, 
wherein the overlay Is synchronized with the video frame based on the timing (fig. 2, 6B, 
col. 6, II. 18-33, col. 10, II. 39-41). 

Regarding claim 78, Kaiser Is silent on comparing the times of data, sleeping for 
a period equivalent to the~difference and awakening for visually Identifying the object in " 
response to an expiration of the time period. Official Notice is taken that use of a 
countdown timer is well known In the art, as is determining the time between events. 
Therefore, it would have been obvious to one of ordinary skill In the art at the time the 
invention was made to modify Kaiser by using a countdown timer and determining the 
time between events in order to set an event in the processor to perform an action at 
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the predicted time of the next event, thereby reducing the processing tasks and 
enabling the processor to perform other functions simultaneously. 

Regarding claim 79, Kaiser teaches actions (as shown in figure 6C-6D), which 
read on an annotation data associated with the object (coL 9, II. 52-65), which is 
associated with the trigger zone (col. 9, II. 29-51 ). Kaiser is silent on removing the data 
structure in response to determining that the item is no longer used. Official Notice is 
taken that removing expired information is well known in the art. Therefore, it would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to modify Kaiser by removing expired information in order to remote the data structure 
from memory thereby conserving memory and efficiently managing the memory 
resources. 

Regarding claim 80, Kaiser is silent on encoding the data structure and image 
overlay data into a packet stream as recited. Official Notice is taken that PIDs are well 
known in the art. Therefore, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to modify Kaiser by using PIDs in order to 
identify one of the plurality of program streams thereby enabling the receiving device to 
decode desirable packets of data. " ~ 

7. Claim 81 is rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. 
Patent 6,61 5,408 to Kaiser et al., U.S. Patent 5,861 ,881 to Freeman et al. and U.S. 
Patent 5,818,438 to Howe et al. in view of U.S. Patent 6,282,713 to Kitsukawa et al. 
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Regarding claim 81 , Kaiser is teaches on encoding the trigger prior to receipt of 
the video (col. 8, II. 14-23), and receiving data structure and the overlay data after the 
receipt of the video at the receiver. Kitsukawa teaches receiving overlay and coupon 
information prior to the receipt of the video (col. 6, II. 51-60). Therefore, it would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
modify Kaiser by receiving the overlay and coupon information prior to receiving the 
video as taught by Kitsukawa in order to provide potential customers with product 
information and incentives to purchase that exploits the convenience of electronic 
purchasing (col. 1, II. 49-55). 

8. Claims 82-89 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
U.S. Patent 6,615,408 to Kaiser etal., U.S. Patent 5,861 ,881 to Freeman et a!., U.S. 
Patent 5,818,438 to Howe et al., and U.S. Patent 6,282,713 to Kitsukawa et al. in view 
of U.S. Patent 6,233,253 to Settle et al. 

Regarding claim 82, Kaiser is silent on a fixed number of packets allocated to 
each of the image overlay data sets, wherein the fixed number of packets corresponds 
to a number of packets filled by the largest image data set. Settle teaches a method for 
inserting NULL packets into an MPEG bitstream. Settle teaches determining the 
multiplexing ratio of the null packets with respect to the video and audio packets (col. 
1 1 , II. 18-35), thereby assigning a fixed number of packets for the data stream, thereby 
the fixed number a based on the overall bitrate of the channel (col. 10, II. 6-35). 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
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invention was made to modify Kaiser by determining the packets to be encoded, 
wherein the number of packets is based on the data to be sent as taught by Settle in 
order to encode at the appropriate multiplexing rate various signals thereby maintaining 
a constant bit-rate signal to the receiver. 

Regarding claim 83, Kaiser is silent on using null packets to fill packets less than 
the fixed number of packets. Settle teaches using null packets to fill the vacant slots 
(col. 4, II. 3-11). Therefore, it would have been obvious to one of ordinary skill in the art 
at the time the invention was made to modify Kaiser by using null packets to fill the 
vacant packet slots as taught by Settle in order to fill the unused data capacity at the 
output thereby maintaining a constant bit rate signal. 

Regarding claim 84, Kaiser is silent on repacking the data stream. Settle 
teaches replacing the null packets with other data packets (col. 10, II. 51-60). 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify Kaiser by replacing the null packets with data packets as 
taught by Settle in order to efficiently distribute data to the receivers. Settle teaches 
ensuring that the buffers are appropriately filled to prevent buffer overflows and 
underflows (col. 1 , II. 45-48), but Kaiser and Settle are silent on packing backwards in 
time. Official Notice is taken that packing data to be received before it is remotely 
processed is well known in the art. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Kaiser by packing 
the data in order to be received before it is needed thereby ensuring that the data can 
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be processed by the receiver thus making the system more robust and readily providing 
the data to the user. 

Regarding claim 85, Kaiser is silent on receiving a packet stream with the video 
stream. Official Notice is taken that the use of data streams in the video stream is well 
known in the art. Therefore, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to modify Kaiser by using data streams in the 
video stream in order to efficiently send data to the user. 

Regarding claim 86, Kaiser teaches receiving the packet stream separately with 
an out-of-band channel (col, 5, II. 46-51). 

Regarding claim 87, Kaiser teaches identifying a first two-dimensional section, 
wherein the region is associated with the object and includes one or more pixels, as 
shown in figure 6B. Kaiser teaches filling the region, moving in a first direction along a 
third dimension (e.g. time) (col. 10, II. 20-41). By moving the highlighted portions with 
the frames, the system of Kaiser determines and fills the different sections of the frames 
with respect to time. Kaiser is silent on returning the original highlighted portion and 
moving backwards to fill the area. Official Notice that preprocessing on filling targets is 
well known in the art. Therefore, it would have been obvious to one of ordinary skill in 
the art at the time the invention was made to modify Kaiser by filling the area before 
display in either direction before playback in order to efficiently playback the video with 
the highlighted regions. 

Regarding claim 88, Kaiser teaches a video frame (col. 10, II. 20-41). 
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Regarding claim 89, Kaiser teaches the overlays moving with respect to time 
(col. 10, 11.20-41). 



Application/Control Number: 09/747,677 Page 13 

Art Unit: 2611 

Conclusion 

9. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Andrew Y Koenig whose telephone number is (703) 
306-0399. The examiner can normally be reached on M-Th (7:30 - 6:30). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Christopher Grant can be reached on (703) 305-4755. The fax phone 
nurtiber for tfTe organization where this application or proceeding is assigned is 703- 
872-9306. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



ayk 




CHRIS GRMir 
PRIMARY EXAMINER 



